DESCRIPTION 



Title of Invention 

VIDEO ENCODER, VIDEO DECODER, VIDEO PROCESSOR AND METHODS 
THEREOF 



Fi Id of the Art 

The present invention relates to a video encoder, video 
decoder, video processor and methods thereof, and more 
particularly to such methods and apparatuses capable of 

fficiently encoding a picture and further obtaining a decoded 
picture in an appropriate form to a monitor for displaying the 
picture . 

Background Art 

Fig. 16 shows a structural example of a video transmission 
system for transmitting a picture from its transmitting side to 
receiving side. 

On the transmitting side, in a high definition video camera 
201, an object is imaged and a high definition picture formed of 

1920X1035 pixels in horizontal by longitudinal in the 16:9 aspect 
ratio (hereinafter, it is preferably referred to as HD picture) is 
outputted, for example. The HD picture for approximately 30 
frames will be normally outputted per second from the video camera 
201. But here the HD picture is subject d to interlaced scanning. 



Ther fore, th HD picture for approximately 60 fields is outputt d 
per second from the video camera 201. 

On the transmitting side, in a standard or low definition 
vid o camera 202, an object is imaged and a standard or low 
d finition picture in the 4:3 aspect ratio (hereinafter, it is 
preferably referred to as SD picture) is outputted, for exampl . 
If the video camera 202 is in the national television system 
committee (NTSC) system or the phase alternation by line (PAL) 

system, one frame is formed of 720X480 pixels or 720X576 pixels 
respectively and the number of frames per second (the number of 
fields) is 30 (60) or 25 (50) respectively, for example. 

On the transmitting side, in a progressive imager 203, an 
object is scanned and the scanned picture (hereinafter, it is 
preferably referred to as progressive picture) is sequentially 
outputted. And on the transmitting side, in a computer 204, a 

picture formed of, e.g., 640X480 pixels is generated and 
outputted as computer graphics (CG) . 

The ratio in horizontal to longitudinal of pixels that 
compose pictures to be outputted by the video cameras 201 and 202 
is approximately 1:1.1. The ratio in horizontal to longitudinal 
of pixels that compose a picture to be outputted by the computer 
204 is 1:1. 

As the above, pictures different in its aspect ratio, the 
number of pixels, its scanned method, the ratio of pixels in 
horizontal and longitudinal, etc., will be outputted from the 
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video cam ras 201 and 202, progressiv imager 203 and comput r 204. 

These pictures are inputted to an editor 205 • In the editor 
205 , pictures from the video cameras 201 and 202 , progressive 
imager 203 and computer 204 are edited respectively. All the 
edited pictures are converted into progressive pictures formed of , 

e.g., 1920X1080 pixels and outputted to a source encoder 206. In 
th source encoder 206, the pictures outputted from the editor 205 
is coded (MPEG-coded) based on the moving picture experts group 
(MPEG) standard for example, and thus obtained coded data is 
supplied to a channel encoder 207 . 

The adoption of that converting all of pictures different 

in the number of pixels, its scanned method, etc., into 1920 X 
1080-pixel progressive pictures by advanced television (ATV) has 
b en planned. 

In the channel encoder 207, channel coding is performed to 
improve the reliability of the coded data in transmission. That 
is, in the channel encoder 207, e.g., error correcting codes 
(ECCs) are added to the coded data as processing for error 
correction, and further it is subjected to prescribed modulation 
or the like. The transmit data obtained by the processing in the 
channel encoder 207 is transmitted via a transmission line 211. 

On the receiving side, the transmit data transmitted from 
the transmitting side as the above is received. This transmit 
data is supplied to a channel decoder 208 to be channel-decoded. 
Specifically, prescribed demodulation is p rformed and furth r 
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error correction using the ECCs or the like is perform d, for 
x ample . 

The coded data obtained as a result of the processing in 
th channel decoder 208 is supplied to a source decoder 209. In 
the source decoder 209, the coded is expanded by, for example, 
decoded (MPEG-decoded) based on the MPEG standard, and thus 
obtained picture data is supplied to a processor 210. 

In the processor 210, the picture data from the source 
d coder 209 is processed to be matched to the format of an output 
d vice to output the picture data. That is, in the case where the 
picture data is displayed in an HD display device 221 for 
displaying HD pictures, in the processor 210, the picture data 
outputted by the source decoder 209 is processed into an 

int rlace-scanned HD picture composed of, e.g., 1920X1035 pixels 
in the 16:9 aspect ratio. In the case where the picture data 
displayed in an SD display device 222 for displaying SD pictures, 
in the processor 210, the picture data outputted by the source 
decoder 209 is processed into an SD picture in the NTSC system or 

the PAL system, composed of 720X480 pixels or 720X576 pixels for 

xample. In the case where the picture data is printed out by a 
printer 223, in the processor 210, the picture outputted by th 
source decoder 209 is converted into a picture of which the ratio 
of pixels in horizontal to longitudinal is corresponded to the 
printer 223. On the other hand, in the case where the picture 
data is displayed on a comput r display 224, in the processor 210, 
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th pictur data outputted by th source d cod r 209 is process d 

into a picture composed of 640X480 pixels for example. 

In the HD display device 221, the SD display device 222, 
th printer 223 and the computer display 224 , the picture from th 
processor 210 is displayed or printed out. 

By the way, heretofore, the editing processing by the 
ditor 205 , the compression processing by the source encoder 206 
and the channel coding processing by the channel encoder 207 on 
the transmitting side have quasi conducted respectively and 
ind pendently. 

For example, the compressed data has less information 
amount than the data before compression processing. Thus, if the 
compressed data can be set to be edited, a load on the editor 205 
can be reduced. However, if the picture is MPEG-coded in the 
source encoder 206 as described above, the bit stream obtained as 
its result becomes difficult to be edited unless in a group of 
picture (GOP) unit, and editing of that is limited to so-called 
cut editing by only connecting the GOPs. Since compression 
processing regardless of the editing processing by the editor 205 
is performed in the source encoder 206, the compressed data cannot 
be edited in a frame unit and it is difficult to give various 
effects on it. 

Moreover, for example, in the source encoder 206, 
compression processing is not performed in consideration of the 
addition of ECCs by the channel ncoder 207. Therefore, for 
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x ample, if ECCs are added to the cod d data obtained by th 
compression processing, sometimes compressibility as the entir 
data after the addition of ECCs has deteriorated. 

As the above, heretofore, since the processing necessary 
for picture coding, e.g., the editing processing, compression 
processing and channel coding processing, etc., has not performed 
in consideration of the other processing, it has been difficult to 
p rform efficient processing. 

Disclosure of Invention 

The present invention is provided considering the above 
asp cts, and it enables to perform efficient processing. 

A video encoder according to Claim 1 is characterized by 
including a processing means for performing one or plural 
processing necessary to encode a picture considering the other 
processing. 

A method for encoding a picture according to Claim 5 is 
characterized by performing one or more processing among the 
plural processing necessary to encode a picture considering the 
oth r processing. 

A video decoder according to Claim 6 is characterized by 
including a generation means for generating a decoded picture 
corresponding to the resolution of an output device for outputting 
the picture by linearly coupling transmit data to prescribed 
coefficients . 
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A method for decoding a pictur according to Claim 7 is 
characterized by generating a decoded picture corresponding to the 
resolution of an output device for outputting the picture by 
lin arly coupling the transmit data to the prescribed coeffici nts. 

A video processor according to Claim 8 is characterized by 
including a processing means for performing one or more processing 
among the plural processing necessary to encode a picture 
considering the other processing, and a generation means for 
gen rating a decoded picture corresponding to the resolution of an 
output device for outputting the picture by linearly coupling the 
data obtained as the result of processing by the processing means 
to the prescribed coefficients* 

A method for processing a picture according to Claim 9 is 
characterized by performing one or more processing among the 
plural processing necessary to encode a picture considering the 
oth r processing, and generating a decoded picture corresponding 
to the resolution of an output device for outputting the picture 
by linearly coupling thus obtained data to the prescribed 
coefficients. 

In the video encoder according to Claim 1, the processing 
means performs one or more processing among the plural processing 
necessary to encode the picture considering the other processing. 

In the method for encoding a picture according to Claim 5, 
one or more processing among the plural processing necessary to 
ncode the picture are performed considering the other processing. 
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In the vid o decoder according to Claim 6, th generation 
m ans generates a decoded picture corresponding to the resolution 
of an output device for outputting the picture by linearly 
coupling transmit data to prescribed coefficients. 

In the method for decoding a picture according to Claim 7 , 
a d coded picture corresponding to the resolution of an output 
d vice for outputting the picture is generated by linearly 
coupling the transmit data to the prescribed coefficients. 

In the video processor according to Claim 8, the processing 
m ans performs one or more processing among the plural processing 
necessary to encode the picture considering the other processing, 
and the generation means generates a decoded picture corresponding 
to the resolution of an output device for outputting the picture 
by linearly coupling the data obtained as the result of processing 
by the processing means to the prescribed coefficients. 

In the method for processing a picture according to Claim 9, 
one or more processing among the plural processing necessary to 

ncode the picture is performed considering the other processing, 
and a decoded picture corresponding to the resolution of an output 
d vice for outputting the picture is generated by linearly 
coupling thus obtained data to the prescribed coefficients. 

Brief Description of Drawings 

Fig. 1 is a block diagram showing an embodiment of a video 
transmission system in which the pres nt invention is applied. 
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Figs. 2A and 2B ar diagrams explaining th ADRC processing. 

Fig. 3 is a block diagram showing a structural example of 
ADRC processing circuit. 

Fig. 4 is a block diagram showing a structural example of 
sync block forming circuit. 

Fig. 5 is a diagram showing the format of a sync block. 

Figs. 6A to 6C are diagrams explaining hierarchical coding. 

Fig. 7 is a block diagram showing a structural example of 
hierarchical encoding circuit. 

Fig. 8 is a block diagram showing a structural example of 
ISDB transmitter. 

Fig. 9 is a block diagram showing a structural example of 
ADRC decoding circuit. 

Fig. 10 is a block diagram showing another structural 
example of ADRC decoding circuit. 

Fig. 11 is a block diagram showing a structural exampl of 
ISDB receiver. 

Fig. 12 is a block diagram showing a structural example of 
resolution creating circuit. 

Fig. 13 is a diagram showing a classif icatory block and a 
pr dictive value calculating block. 

Figs. 14A and 14B are diagrams explaining classification 
processing. 

Fig. 15 is a block diagram showing a structural example of 
for casting circuit 134 of Fig. 12. 
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Fig. 16 is a block diagram showing a structural xampl of 
video transmission system for transmitting a picture from its 
transmitting side to receiving side. 

Best Mode for carrying Out the Invention 

An embodiment of the present invention will be describ d in 
detail with reference to the accompanying drawings. 

Fig. 1 shows a structural example of the embodiment of video 
transmission system in which the present invention is applied. 

On its transmitting side, pictures composed of the maximum 
number of pixels capable of being outputted by an extended definition 
vid o camera 1, a video camera 2 in low definition, etc., are 
outputted from them. Specifically, for example, a progressive HD 

picture which is composed of 1920X960 pixels, and of which the 
aspect ratio is 16:9 and the frame rate is approximately 30 frame/s c, 
is outputted from the video camera 1. And for example, a progressive 

SD picture which is composed of 640X480 pixels, and of which the 
aspect ratio is 4:3 and the frame rate is approximately 30 frame/s c, 

or a progressive SD picture which is composed of 720X480 pixels, and 
of which the aspect ratio is the 4:3 and the frame rate is 
approximately 15 frame/sec is outputted from the video camera 2. 

On its receiving side, a progressive imager 3 scans an obj ct 
and outputs for example, a progressive picture of which the 
longitudinal number of pixels is integral times of 480. And a 
computer 4 generates and outputs for example, a picture which is 
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composed of 640X480 pixels and of which a ratio in horizontal to 
longitudinal is 1:1, as computer graphics (CG) . 

Such pictures outputted by the video cameras 1 and 2, 
progressive imager 3 and computer 4 are supplied to an integrative 
ncoding system 6 (the processing means). 

In addition, for example, a progressive picture of which the 
longitudinal number of pixels is integral times of 480 is supplied 
from a network 5, e.g., an internet or the like, to the integrative 
ncoding system 6. From the network 5, a picture of which the ratio 
in horizontal to longitudinal of each pixel is 1:1 is supplied 
similarly to the computer 4. 

Here, the video cameras 1 and 2 are made to output the 
pictures composed of the maximum number of pixels capable of being 
outputted by them, because if aiming at such pictures for the 
processing, decoded pictures superior in image quality can be 
g nerally obtained comparing with the case of aiming at pictures 
having less number of pixels in one picture, e.g., pictures in an 
interlace system. 

Moreover, all the longitudinal number of pixels of the 
pictures to be supplied to the integrative encoding system 6 is the 
pr scribed value, i.e., integral times of 480 here for example. And 
th ir frame rate is also the prescribed value, integral times of 15. 
This is because when an SD picture is generated by thinning out the 
pix Is forming an HD picture in the spatial direction or temporal 
dir ction or when an HD picture is generated by interpolating the 
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pixels forming an SD picture in th spatial dir ction or temporal 
dir ction, the deterioration in the image quality of the SD picture 
or HD picture to be generated can be reduced. 

The longitudinal number of pixels of the HD picture to b 
outputted to the video camera 1 is set as 960 pixels, because wh n 
the longitudinal number of pixels is integral times of 480, "960" 
most approximates to "1035" being the longitudinal number of pix Is 
of xisting HD pictures (Fig. 16), and thus, the deterioration in its 
image quality can be reduced when in performing crossing over. 

The longitudinal number of pixels of the SD picture to be 
outputted to the video camera 2 is set as 480 pixels, because that 
value is the integral number of 480 most approximate to the number of 
pixels adopted in the present NTSC system or PAL system or the like. 

If the horizontal number of pixels of the SD picture of which 
the aspect ratio is 4:3 to be outputted to the video camera 2 is s t 
to 640, the ratio in horizontal to longitudinal of the pixel becomes 

1:1 (= 4X480:3X640). As a result, compatibility with the pictur to 
be outputted by the computer 4 or the picture to be supplied from the 
network 5 is apt to be obtained. 

If the horizontal number of pixels of the SD picture of which 
th aspect ratio is 4:3 to be outputted to the video camera 2 is set 
to 720, the ratio in horizontal to longitudinal of the pixel becomes 
8:9 (= 4X480:3X720). This is equal to the ratio in horizontal to 
vertical (8:9 = 16X960:9X1920) of the pixels that form the HD 
picture to be outputted by the video camera 1. Therefor , in this 
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cas , if the horizontal and vertical numb rs of pixels of the SD 
picture to be outputted by the video camera 2 are doubled and th 
longitudinal number of pixels is set to 960 pixels that is equal to 
th longitudinal number of pixels of the HD picture to be outputted 
by the video camera 1, quasi roundness can be kept (that is, the 
pictures can avoid being long in the horizontal direction or the 
v rtical direction). 

In the integrative encoding system 6, one or more processing 
n cessary for its encoding, e.g., editing processing, compression 
pressing, channel coding processing or the like, are performed on 
pictures (digital video signals) supplied thereto considering the 
oth r processing. Here, the editing processing includes not only 
quasi video editing processing in which cut editing is performed or 
effects are given but also adding information to realize an 
integrated services digital broadcasting (ISDB: it is interactive 
broadcasting that digitalizes and transmits various information) and 
information to attach other values added (for example, information 
n c ssary to obtain a decoded picture in further high image quality), 
and linking a certain picture to another picture, and the like, for 
example . 

Transmit data obtained by the processing in the integrativ 
ncoding system 6 is transmitted to the receiving side via a 
transmission line 13. As the transmission line 13, other than 
communication lines such as a satellite line, terrestrial wave, CATV 
n twork, public network, internet, etc., the process of magnetic 
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recording/ r producing and also recording mediums such as a magn tic 
disk, optical disk, magnetic tape, magneto-optical disk, and oth r 
recording mediums are included. 

The transmit data transmitted via the transmission line 13 is 
rec ived on the receiving side and supplied to an adaptive decoding 
system 7 (the generation means). The adaptive decoding system 7 is 
connected to output devices for outputting a picture, e.g., a high 
definition display unit 9 for displaying the HD picture, a display 
unit 10 in standard definition or low definition for displaying the 
SD picture, a printer 11 for printing out a picture, and a comput r 
display 12 connected to a computer (however, the printer 11 is 
connected via a converter for aspect ratio of pixel 8). The adaptive 
decoding system 7 generates decoded pictures corresponding to the 
respective resolution of the display units 9 and 10, printer 11 and 
computer display 12 and outputting these pictures to them 
respectively, by linearly coupling the transmit data to the 
prescribed coefficients. 

In the display units 9 and 10 and the computer display 12, the 
decoded picture from the adaptive decoding system 7 is display d, and 
in the printer 11 the decoded picture from the adaptive decoding 
system 7 is printed out. 

Note that, the difference between the ratio in horizontal to 
longitudinal of each pixel in the display units 9 and 10 and the 
computer display 12 (hereinafter, it is preferably referred to as 
asp ct ratio of pixel) and th aspect ratio of pix 1 forming the 
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d coded picture is absorbed by horizontal scanning in each of th 
display units 9 and 10 and the computer display 12. 

On the other hand, in the printer 11, since the differenc of 
the aspect ratio of pixel cannot be absorbed by such horizontal 
scanning, the converter for aspect ratio of pixel 8 is provided in 
the preceding stage* There the aspect ratio of pixel of the decod d 
picture is converted into a value applicable to the printer 11. 

Then, processing in the integrative encoding system 6 will be 
described. 

In the integrative encoding system 6, for example, the 
compression processing is performed considering the editing 
processing. 

In the integrative encoding system 6, an adaptive dynamic 
range coding (ADRC) processing is performed as the compression 
processing, for example. 

Here, the ADRC processing will be described briefly. 

To simplify the description, if considering a block compos d 
of four pixels in line, in the ADRC processing, the maximum value MAX 
and the minimum value MIN of their pixel values are detected as shown 
in Fig. 2A. And DR = MAX - MIN is set as the local dynamic range of 
the block, and the pixel values of the pixels forming the block are 
requantized into K bit. 

Specifically, the minimum value MIN is subtracted from each 
pixel value in the block. The subtracted values are divided into 
DR/2 K . And th pixel values are converted into codes (ADRC codes) 
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corr sponding to thus obtained divided values. More specif ically, if 
assuming K = 2 for example, it is determined that the divided valu s 
belong to which range among ranges obtained by dividing the dynamic 
range DR into four (= 2 2 ) as shown in Fig. 2B. If the divided valu 
belongs to the range of, e.g., the lowest level, the second level 
from low, the third level from low, or the top level, it is coded 
into two bits, e.g., 00B, 01B, 10B or 11B (B shows a binary numb r) . 

In the ADRC, a smaller value than the number of bits assigned 
to the pixel is used as the number of bits K at the time of 
r quantizing. Therefore, each pixel is compressed into such small 
number of bits (however, the minimum value MIN and the dynamic range 
DR are generated for each block other than that) . 

Note that, its decoding can be performed by converting the 
ADRC code 00B, 01B, 10B or 11B into, e.g., the central value L 00 of 
the lowest level range obtained by dividing the dynamic range DR into 
four, the central value L 01 of the second level range from low, the 
c ntral value L 10 of the third level range from low or the central 
value L u of the top level range, and adding the minimum value MIN to 
the value. 

Fig. 3 shows a structural example of ADRC processing circuit 
that performs the ADRC processing. 

Video data is supplied to a blocking circuit 21 and divided 
there into blocks in prescribed size. That is, the blocking circuit 
21 divides the video data into blocks of , e.g., horizontal 4 by 
longitudinal 4 pixels, and supplies these blocks to a minimum value 
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d tecting circuit: 22, a maximum value det cting circuit 23 and a 
computing unit 25 . 

In the minimum value detecting circuit 22, the minimum value 

MIN is detected from among 16 (= 4X4) pixels forming the block from 
th blocking circuit 21. This minimum value MIN is outputted as one 
of the signals obtained by the ADRC processing as well as supplied to 
a computing unit 24 and the computing unit 25. 

At the same time, in the maximum value detecting circuit 23, 
th maximum value MAX is detected from among 16 pixels forming the 
block from the blocking circuit 21 and supplied to the computing unit 
24. 

In the computing unit 24, the minimum value MIN is subtracted 
from the maximum value MAX to obtain the dynamic range DR of the 
block. This dynamic range DR is outputted as one of the signals 
obtained by the ADRC processing as well as supplied to a quantizing 
circuit 26. 

In the computing unit 25, the minimum value MIN of the block 
is subtracted from each of the 16 pixels forming the block, and th 
subtracted values are supplied to the quantizing circuit 26. In the 
quantizing circuit 26, the subtracted values from the computing unit 
25 are quantized in a quantization step corresponding to the dynamic 
range DR from the computing unit 24. That is, in the quantizing 
circuit 26, for example, the output of the computing unit 25 is 
divided into DR/2 K , and a value which the decimal fractions has 
omitted is outputted as an ADRC code CODE (requantized result of 
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pixel) . 

In this embodiment, the number of bits K in requantizing is 
det rmined, for example, corresponding to the size of the dynamic 
range DR of each block and the ADRC code is variable length. 

For example, it is assumed that four threshold values Tl, T2, 
T3 and T4 are now set and their size relationship is 0 < Tl < T2 < T3 
< T4 and also eight bits are assigned to the pixels of an original 
picture (thus, T4 is less than 2 8 ) . 

In this case, in the quantizing circuit 26, it is determined 
wh ther or not the dynamic range DR is within one of 0 or more and 
less than Tl, Tl or more and less than T2, T2 or more and less than 
T3, T3 or more and less than T4 and T4 or more and less than 2 8 . And 
if the dynamic range DR is within one of those, for example, 0 
through 4 bits are respectively assigned as the number of bits K in 
requantizing. Therefore, in this case, the ADRC code becomes 0 bit 
at the minimum and 4 bit at the maximum. 

When the ADRC code is variable length, the number of bits K of 
the ADRC code is necessary for its decoding* Therefore, the 
quantizing circuit 26 outputs a threshold code showing that the 
dynamic range DR is within which range among the above ranges. This 
threshold code is outputted as the result of ADRC processing togeth r 
with the minimum value MIN, dynamic range DR and ADRC code. 

Note that, the minimum value MIN, dynamic range DR and 
threshold code are set in variable length for example. 

In the above case, the ADRC cod is set in variable length, 
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howev r, th ADRC cod can be s t in fixed length provid d that the 
number of bits K in requantizing is set to a fixed value irrespective 
of the dynamic range DR of the block. 

The ADRC code obtained by the above ADRC processing becomes 
less than the number of bits assigned to the original pixels. On th 
other hand, the minimum value MIN, dynamic range DR and ADRC code 
obtained by the ADRC processing can be used in a block unit, thus the 
ADRC-processed picture can be edited in a frame unit for example. 

From the above, by conducting the ADRC processing as the 
compression processing and setting the result of the ADRC processing 
as an object of various editing processing, the almost same editing 
processing as the case where the original picture before ADRC 
processing is set an object can be performed, and at the same time, a 
load of the processing can be reduced comparing with the case where 
th original picture before ADRC processing is set as the object. 

Thus it can be said that the ADRC processing as compression 
processing is performed considering the editing processing, and the 
editing processing can be executed efficiently. 

The detail of the ADRC has been disclosed in the patent 1 aid- 
op n No. Hei3 (1991)-53778 previously filed by the present applicant, 
or the like, for example. 

The ADRC can be referred to as block coding since it performs 
coding in a block unit. In addition to the ADRC, however, the block 
coding includes coding which obtains the mean value and the standard 
d viation of pix Is forming a block and also 1-bit flag showing th 
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siz relationship betwe n each pix 1 and th mean value, and th like. 
Such block coding may be adopted as the compression processing by the 
int grative encoding system 6. 

In the above case, the minimum value MIN and the dynamic range 
DR are included in the result of the ADRC processing. In the result 
of the ADRC, also, the minimum value MIN and the maximum value MAX of 
the block, or the dynamic range DR and the maximum value MAX of the 

s 

block can be included in addition to that. 

Moreover, in the above case, a block is composed of one frame 
of horizontal 4 pixel by longitudinal 4 pixel. However, the block 
can be composed of pixels forming plural frames that temporary 
continue . 

By the way, as a method for transmitting the minimum valu MIN, 
dynamic range DR, threshold code and ADRC code for each block 
obtained by the ADRC processing, for example, there is a method which 
forms a block disposes a prescribed amount of data of the ADRC result 
(h reinafter, it is preferably referred to as sync block) following a 
synchronization pattern for matching and performing transmission in 
such sync block unit. 

When transmission is performed in a sync block unit, the 
minimum value MIN, dynamic range DR and threshold code are fixed 
1 ngth as the above. Therefore, if these data are disposed at fixed 
positions in a sync block, even if one sync block cannot be obtained 
by failure, it does not affect minimum values MIN, dynamic ranges DR 
and thr shold codes disposed in other sync blocks. 
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Since the ADRC code is variable length, howev r, in the case 
wh re an ADRC code is separately disposed in plural sync blocks 
b cause it cannot be contained in one sync block for example , th 
failure of the one sync block sometimes affects the other sync blocks. 
Sp cifically, if the first sync block has failed among the plural 
sync blocks , it is unknown that the ADRC code disposed at the top of 
th second sync block is corresponding to which position's pixel in 
th block , further it is unknown that the bit disposed as the ADRC 
cod is whether a bit forming the following (a part of) ADRC code 
disposed at the end of the first sync block or the first bit of the 
ADRC code. As a result, even an ADRC code disposed in a sync block 
after the second cannot be fetched. As the above, a certain sync 
block error propagates to the other sync blocks. 

By the way, even if all the ADRC codes of a certain block has 
lost, if the minimum value MIN has known, the block in which all the 
pix Is have the minimum value MIN as a pixel value can be reproduced. 
However, since this block has the same pixel value, practically flat, 
th reproducibility of the original picture is low. 

To realize its further higher reproducibility, it can be 
considered that the ADRC code is separated into, for example, th 
most significant bit (MSB) and others (hereinafter, it is preferably 
r f rred to as remaining bits) and disposing also the MSB at a fixed 
position in the sync block similarly to the minimum value MIN, 
dynamic range DR and threshold code. In this case, even if the 
r maining bit has lost, th block formed of binary can be obtained by 
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invers ly quantizing th MSB based on the dynamic range DR. Thus, 
the picture with higher reproducibility can be obtained comparing 
with the case where all the ADRC codes have lost. 

Fig. 4 shows a structural example of sync block forming 
circuit that performs sync block processing to form the above sync 
block . 

The minimum value MIN, dynamic range DR and threshold code 
outputted from the ADRC processing circuit (Fig. 3) are supplied to a 
multiplexer 32 and the ADRC code CODE is supplied to a separator 31. 
In the separator 31, the ADRC code is separated into the MSB and the 
remaining bits and both are supplied to the multiplexer 32 . 

The synchronization pattern has supplied to the multiplex r 32 
in addition to the above data. The multiplexer 32 performs time 
division multiplexing on the data supplied thereto and forms and 
outputs a sync block such as shown in Fig. 5. 

That is, as shown in Fig. 5, the synchronization pattern 
having a fixed length is disposed at the top of the sync block, and 
added data having a fixed length is disposed following it. Here, the 
added data is formed only fixed length data such as the threshold 
cod or the like. Following the added data, the remaining bits are 
disposed by a prescribed number of bytes, that is, after the add d 
data, the remaining bits are disposed from the head to an N^l-th 
byte . 

The dynamic range DR, MSB and minimum value MIN are disposed 
after from th top to an N x -th byte in the order of DR, MSB, MIN, MSB, 
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DR, . . . , for xampl . Following th prescrlb d number of the 
disposed dynamic range DR, MSB and minimum value MIN, the remaining 
bits are disposed again. Then, after an N 2 -th byte from the top, th 
prescribed number of the dynamic range DR, MSB and minimum value MIN 
are disposed again in the aforementioned order. Hereinafter, th 
similar disposition is repeated until the end of the sync block. 

Since the dynamic range DR, the MSB and the minimum valu MIN 
are disposed from the determined position as from the N x -th byt , N 2 - 
th byte , ... from the head of the sync block as the above and these 
data are fixed length, they are disposed at the fixed position in the 
sync block. 

In the above sync block processing, even if an error is caused 
in the ADRC code (remaining bits), a decoded picture comparatively 
close to the original picture can be obtained. Furthermore, even if 
no bit remains, a decoded picture in high reproducibility can be 
obtained. Thus, extremely to say, it is unnecessary to add an ECC 
for error correction to the remaining bits, for example. In this 
cas , a load against channel coding processing can be reduced. For 
this reason, it can be said that the sync block processing is 
p rformed considering channel coding processing. 

■ 

Note that, the detail of the sync block processing has be n 
disclosed in the patent laid-open No. 2 ( 1990) -162980 previously filed 
by the present applicant or the like. 

The integrative encoding system 6 can also perform 
hi rarchical coding processing as the compression processing in place 
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of th ADRC processing, for xample. 

In the hierarchical coding, for example, high resolution 
picture data is set as the picture data of the lowest hierarchy or 
th first hierarchy, and the picture data of the second hierarchy 
(compressed picture) of which the number of pixels is less than the 
first hierarchy's is generated and the picture data of the third 
hierarchy of which the number of pixels is less than the second 
hierarchy's is generated. Similarly, picture data are generated 
until the most upper hierarchy. The picture data of each hierarchy 
is displayed on a monitor having a resolution (the number of pix Is) 
corresponding to the hierarchy. Thus, on a user side, a picture of 
th same contents can be viewed by selecting picture data 
corresponding to the resolution of his monitor from among the 
hierarchically-coded picture data. 

By the way, in the case where picture data having a certain 
resolution is set as the picture data of the lowest hierarchy (first 
hierarchy) and the picture data of an upper hierarchy is sequentially 
formed and all of them are stored or transmitted as they are, much 
storage capacity or transmission capacity is required by the pictur 
data of the upper hierarchy comparing with the case of storing only 
the picture data of the lowest hierarchy. 

For that reason, here, a hierarchical coding without such 
increasing of storage capacity or the like is adopted as the 
compression processing in the integrative encoding system 6. 

For xample, now the m an value of four pixels, 2X2 
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(horizontal by vertical) pix Is in a lower hierarchy is s t as th 

pix Is (pixel value) of an upper hierarchy, and performing 

hi rarchical coding in three hierarchies. In this case, considering 

8X8 pixel as the picture of the lowest hierarchy as shown in Fig. 6A, 
th mean value mO of four pixels hOO, hOl, h02 and h03 at the upper 
1 ft of the second hierarchy is operated and set as the one pixel of 
th upper left of the second hierarchy. Similarly, the mean valu ml 
of four pixels hlO, hll, hl2 and hl3 at the upper right of the 
picture of the lowest hierarchy, the mean value m2 of four pixels h20, 
h21, h22 and h23 at the lower left and the mean value m3 of four 
pix Is h30, h31, h32 and h33 at the lower right are operated and set 
as the one pixel at the upper right, lower left and lower right of 
the second hierarchy respectively. Furthermore, the mean value (q) 

of 2X2 pixels of the second hierarchy, the four pixels mO, ml, m2 
and m3, is operated and set as the pixel of the picture of the third 
hi rarchy, i.e., the most upper hierarchy, here. 

If all the above pixels hOO to h03, hlO to hl3, h20 to h23, 
h30 to h33, mO to m3 and (q) are stored as they are, much storage 
capacity or the like is required by the pixels mO to m3 and (q) as 
described above. 

Then, as shown in Fig. 6B, the pixel (q) of the third 
hi rarchy is disposed at, e.g., the position of the pixel m3 at the 
lower right among the pixels mO to m3 of the second hierarchy. Thus, 
th second hierarchy is composed of the pixels mO to m2 and (q) . 

As shown in Fig. 6C, th pixel mO of the second hi rarchy is 
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disposed at, e.g., the position of th pix 1 h03 at the lower right, 
among the pixels hOO to h03 of the third hierarchy that has used to 
obtain mO. Similarly, the remaining pixels ml, m2 and (q) of th 
s cond hierarchy are disposed in place of the pixels hl3, h23 and h33 
of the first hierarchy. Note that, the pixel (q) has not obtained 
directly from the pixels h30 to h33 but since it has been dispos d in 
the second hierarchy in place of m3 obtained directly from them, the 
pix 1 (q) is disposed instead of disposing the pixel m3 at the 
position of the pixel h33. 

By conducting as the above, as shown in Fig. 6C, the total of 

pixels becomes 16 pixel of 4X4. This is the same as the case of 
only the pixels of the lowest hierarchy shown in Fig. 6A. Therefore, 
in this case, an increase of storage capacity or the like can be 
prevented . 

In this connection, the decoding of the pixel m3 changed into 
the pixel (q) and the pixels h03, hl3, h23 and h33 changed into the 
pix Is mO to m3 respectively can be performed as follows . 

Since (q) is the mean value of mO to m3, the equation q = 
(m0+ml+m2+m3)/4 is satisfied. Thus, m3 can be obtained by the 

qua t ion m3 = 4 X q- (m0+ml+m2 ) . 

Since mO is the mean value of hOO to h03, the equation mO = 
(h00+h01+h02+h03)/4 is satisfied. Thus, h03 can be obtained by the 

equation h03 = 4XmO- (h00+h01+h02) . In similar manner, hl3, h23 and 
h33 can be obtained. 

Fig. 7 shows a structural exampl of hierarchical coding 
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circuit that performs the abov hierarchical coding proc ssing. In 
this hierarchical coding circuit, the aforementioned hierarchical 
coding in three hierarchies is performed for example. 

The first hierarchy (lowest hierarchy) picture data (her , it 
is progressive picture as described above) is supplied to a mean 
value calculating circuit 41 and a pixel extracting circuit 43. 

In the mean value calculating circuit 41 , with respect to th 

first hierarchy picture, for example, the mean value of 2X2 pixels 
the total four pixels as the above is computed, and the second 
hierarchy picture is generated. This second hierarchy picture is 
supplied to a mean value calculating circuit 42 and a pixel 
extracting circuit 44. 

In the mean value calculating circuit 42, with respect to the 

s cond hierarchy picture, for example, the mean value of 2X2 pixels 
the total four pixels is computed, and the third hierarchy pictur is 
g nerated. This third hierarchy picture is supplied to a pixel 
inserting circuit 45 . 

In the pixel extracting circuit 43, pixels which correspond to 
the pixels h03, hl3 and h23 described in Fig. 6 are extracted from 
the first hierarchy picture and the remains are supplied to the pixel 
inserting circuit 45. In the pixel extracting circuit 44, pixels 
which correspond to the pixel m3 described in Fig. 6 is extracted and 
the remains are supplied to the pixel inserting circuit 45. 

In the pixel inserting circuit 45, the pixels of the second 
hi rarchy picture (e.g., the pixels mO to m2) from the pixel 
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extracting circuit 44 are insert d to positions corr sponding to the 
pixels h03, hl3 and h23 of the first hierarchy picture from the pix 1 

xtracting circuit 43 , and the pixel of the third hierarchy picture 
( .g. , pixel (q)) from the mean value calculating circuit 42 is 
ins rted to a position corresponding to the pixel h33 of the first 
hi rarchy picture. In the above manner, the picture data described 
in Fig. 6C is formed and this is outputted as the result of the 
hi rarchical coding. 

If according to normal hierarchical coding, much storage 
capacity or transmission capacity is required by the picture data of 
upp r hierarchies, however, according to the hierarchical coding 
described in Figs. 6 and 7 (hereinafter, it is preferably referred to 
as improved hierarchical coding) , the amount of data to be obtained 
as the result is the same as the picture of the lowest hierarchy. 
For this reason, the improved hierarchical coding can be said to be 
information compressing processing. 

When the hierarchical coding is performed, the picture of a 
lower hierarchy can be obtained by performing , for example , 
int rpolation or the like using an upper hierarchy picture (howev r, 
thus obtained picture is not the same picture as the lower hierarchy 
picture but a picture deteriorated in image quality.) Therefore , 

v n in the worst, the pictures of all the hierarchies can be 
obtained provided that the picture of the most upper hierarchy can be 
restored, so that, for example , the addition of an ECC for error 
correction is sufficient only performing to th picture of the most 



28 



4 

I 



upper hierarchy and it is unnecessary to p rform to the pictures of 
all the hierarchies. In this case a load to channel coding 
processing can be reduced. For this reason, it can be said that th 
hi rarchical coding processing is performed considering the channel 
coding. 

Note that, in the above case, the upper hierarchy pictur is 
gen rated with reducing the number of pixels in the spatial direction, 
but the upper hierarchy picture can be generated with reducing the 
number of pixels in the temporal direction for example. 

Then, it will be described about the addition of information 
for realizing an ISDB, one of the editing processing in the 
integrative encoding system 6. 

Fig. 8 shows a structural example of ISDB transmitter being a 
part of the integrative encoding system 6 that realizes the ISDB. 

For example, an SD picture and audio attached thereto ar 
inputted to an encoding part 51, and they are subjected there to th 
compression processing as the aforementioned ADRC processing. Thus 
obtained signal is outputted to a multiplexing part 57. Furthermore, 
the encoding part 51 outputs a synchronizing signal which repres nts 
th timing of the compression processing to a time code generating 
part 52. The time code generating part 52 generates a time cod or 
the like as additive information to be added to the output of the 
ncoding part 51 synchronizing with the synchronizing signal from the 
ncoding part 51 and outputs this to the multiplexing part 57. 

Also in an encoding part 53 or a time code generating part 54, 
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the processing similar to th encoding part 51 or th time code 
gen rating part 52, except for that an object of the processing is 
not SD picture but HD picture, is performed respectively. Both of 
th coded data obtained by the compression processing in the encoding 
part 51 and the time code outputted by the time code generating part 
54 are supplied to the multiplexing part 57. 

To an encoding part 55, for example, a computer program, data 
necessary to execute the program, facsimile data and also data for 
realizing multimedia such as local information are inputted. These 
data are compressed there and outputted to the multiplexing part 57 
as coded data. Furthermore, the encoding part 55 outputs the 
synchronizing signal which represents the timing of the compression 
processing to an additive information generating part 56. The 
additive information generating part 56 generates additive 
information which represents a type of the data compressed in the 
encoding part 55, synchronizing with the synchronizing signal from 
th encoding part 55 and outputs this to the multiplexing part 57 . 

Here, the local information to be inputted to the encoding 
part 55 is information peculiar to each area, and it includes, for 
example, a weather forecast, map, information on establishments ( . g., 
th service contents and business hours of restaurants, etc.) and 
adv rtisements in each area. With respect to such local information, 
th additive information generating part 56 generates an area code 
which represents an area corresponding to each local information as 
added information. 
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In the multiplexing part 57, the outputs of th encoding part 
51, time code generating part 52, encoding part 53, time code 
gen rating part 54, encoding part 55 and additive information 
g n rating part 56 are multiplexed and outputted. 

s 

Then, Fig. 9 shows a structural example of the adaptive 
decoding system 7 (Fig. 1) in the case where the ADRC processing is 
performed as the compression processing in the integrative encoding 
system 6. That is, Fig. 9 shows a structural example of ADRC 
d coding circuit being a part of the adaptive decoding system 7 that 
performs ADRC decoding processing when the result of the ADRC 
processing is decoded. 

To a demultiplexer 101, a bit stream in which the result of 
the ADRC processing is disposed is inputted as transmit data 
transmitted via the transmission line 13 (Fig. 1), and there the 
minimum value MIN, dynamic range DR and ADRC code are separated from 
the transmit data. Note that, in the demultiplexer 101, the 
separation of the ADRC code is performed by separating the threshold 
code from the transmit data and recognizing the number of assign d 
bits to the ADRC code (the above K) based on the threshold code. 

The minimum value MIN is supplied to an arithmetic unit 103, 
and the dynamic range DR and the ADRC code are supplied to an inv rs 
quantizing circuit 102, respectively. In the inverse quantizing 
circuit 102, the ADRC code is inversely quantized in a quantization 
step corresponding to the dynamic range DR, and thus obtained 
inv rsely-quantized value is supplied to the arithm tic unit 103. In 



th arithmetic unit 103 , th inversely-quantized value from the 
inverse quantizing circuit 102 is added to the minimum value MIN. 
Th reby, the pixels are decoded. 

If obtaining a pixel for one block, the arithmetic unit 103 
supplies this to a frame forming circuit 104. The frame forming 
circuit 104 sequentially stores the pixels which have supplied in a 
block unit and outputting them every storing of it. 

In the case where the ADRC code has separated into the MSB and 
th remaining bits as described above, the demultiplexer 101 also 
p rforms processing for storing the original ADRC code by combining 
the MSB with the remaining bits. Furthermore, in the case where th 
ADRC code has separated into the MSB and the remaining bits, if an 
error has occurred in the remaining bits, the demultiplexer 101 
outputs the MSB to the inverse quantizing circuit 102 as the ADRC 
code • 

By the way, in the ADRC decoding, even if the remaining bits 
are in error, if the MSB, minimum value MIN and dynamic range DR 

xist, a decoded picture having fine reproducibility in a certain 
d gree (decoded picture similar to the original picture) can be 
obtained, as described above. However, if the minimum value MIN or 
the dynamic range DR is in error, it becomes difficult to decode the 
block . 

Fig. 10 shows a structural example of ADRC decoding circuit 
capable of decoding a block with relatively fine precision even if 
the minimum value MIN or the dynamic rang DR is in error. Note that, 
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in Fig. 10, the same referenc numerals are added to corresponding 
parts of Fig. 9, and its description will be omitted hereinafter. 
That is, this ADRC decoding circuit is basically similarly configured 
to Fig. 9 except that selectors 105 and 106, a memory 107 and a 
restoring circuit 108 are newly provided. 

To the selector 105, the dynamic range DR outputted by th 
demultiplexer 101 and a predicted value of the dynamic range DR' 
outputted by the restoring circuit 108 are supplied. To the selector 
106, the minimum value MIN outputted by the demultiplexer 101 and a 
pr dieted value of the minimum value MIN' outputted by the restoring 
circuit 108 are supplied. The demultiplexer 101 here detects whether 
an error is occurred in the minimum value MIN and the dynamic range 
DR included in the transmit data, and if an error is caused, it 
outputs an error signal to the selectors 105 and 106. 

The selector 105, when not receiving the error signal, that is, 
no error is caused in the minimum value MIN and the dynamic rang DR, 
s lects the dynamic range DR outputted by the demultiplexer 101 and 
outputs this to the inverse quantizing circuit 102. Similarly, the 
s lector 106, when not receiving the error signal, selects the 
minimum value MIN outputted by the demultiplexer 101 and outputs this 
to the arithmetic unit 103. 

Therefore, in this case, the ADRC decoding processing is 
performed similar to the case of Fig. 9. 

On the other hand, the decoded values of the pixels outputted 
by the arithmetic unit 103 are supplied to not only the frame forming 
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circuit 104 but also the memory 107. In the memory 107 , th decoded 
values of the pixels from the arithmetic unit 103 are stored in each 
corresponding address. 

Then, in the restoring circuit 108, the decoded values of 
pix Is in the circumference of a block now being an object of the 
ADRC decoding processing are read out from the memory 107 by the same 
number as the number of pixels forming a block, that is, 16 pieces in 
this embodiment as described above. Furthermore, the restoring 
circuit 108 detects the minimum value and the dynamic range 
(difference between the maximum value and the minimum value) of that 
sixteen pixels, and outputs each of them to the selectors 106 and 105 
r spectively as the predicted value of the minimum value MIN V and th 
predicted value DR ■ of the dynamic range of the block now being the 
obj ct of the ADRC decoding. 

The selector 105 or 106, if receiving the error signal from 
the demultiplexer 101, that is, an error is caused in the minimum 
value MIN or the dynamic range DR, selects the predicted value of the 
dynamic range DR* or the predicted value of the minimum value MIN ' 
from the restoring circuit 108, and outputs them to the inverse 
quantizing circuit 102 or the arithmetic unit 103 respectively. 

Thus, in this case, in the inverse quantizing circuit 102, 
inverse quantization is performed using the predicted value of the 
dynamic range DR' , and in the arithmetic unit 103, the pixels ar 
d coded using the predicted value of the minimum value MIN' • 

Marking a certain block, normally a larg relationship xists 
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b twe n pix Is forming th marked block and pixels in the 
circumference of the marked block. Therefore, according to the 
pixels having such correlation, the dynamic range and the minimum 
value of the marked block can be predicted in relatively fine 
pr cision. As a result, using thus predicted values enable to obtain 
a d coded picture almost similar to the case of using the true 
minimum value MIN and dynamic range DR. 

The detail of the above ADRC decoding processing has been 
disclosed in the patent laid-open No. S63 (1988) -257390 previously 
filed by the present applicant, for example. 

When the ADRC processing is performed, even if an error has 
caused in the minimum value MIN or the dynamic range DR a decoded 
picture in a certain degree can be obtained as described above. 
Moreover, by performing sync block processing in addition to the ADRC 
processing, to cope with an error in the remaining bits is enabled as 
described above. Furthermore, also in the case of performing 
hi rarchical coding, it is sufficient to perform the processing for 
error correction to the picture of the most upper hierarchy at 1 ast 
as described above, and it is not required to always perform the 
processing to the pictures of all the hierarchies. 

* 

For this reason, the ADRC processing, sync block processing 
and hierarchical coding processing can be said that quasi robust 
processing strongly resistant to errors. Now if such robust 
processing having resistance to errors is assumed as robust coding, 
the processing performed in the integrative encoding system 6 can be 
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said integrative coding processing in which such robust coding and 
diting processing and the like are integrated. 

Among the robust coding, for example, in the ADRC processing, 
an amount of information is reduced and also a resistance to errors 
is improved by its execution. Therefore it can be said that in the 
ADRC processing, the compression processing of pictures and the 
processing for error correction are performed by organically combin d. 

Then, Fig. 11 shows a structural example of ISDB receiver 
being a part of the adaptive decoding system 7 that realizes an ISDB. 

To a signal separating part 111, the transmit data transmitted 
via the transmission line 13 is inputted, and there coded data 
obtained by coding video (SD picture or HD picture) and audio 
accompanying with that for example, are extracted from the transmit 
data. The coded data generated in the signal separating part 111 are 
outputted to a decoding part 112 respectively. 

In the decoding part 112, the coded data from the signal 
separating part 111 are decoded. That is, ADRC decoding or the like 
is performed on the coded video data for example. The video and th 
voice accompanying with that obtained by the decoding processing in 
the decoding part 112 are outputted via a selector 113. The picture 
is supplied to the display device 9 or 10 or the computer display 12 
and displayed, or supplied to the printer 11 via the converter for 
aspect ratio of pixel 8 and printed out. On the other hand, the 
audio is supplied to a speaker (not shown) and outputted. 

In the signal separating part 111, the local information and 
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th ar a cod as added information corresponding to the local 
information are extracted and supplied to a decoding part 114. In 
th decoding part 114, the local information is decoded corresponding 
to the area code. 

That is, since it is not limited that the received local 
information is the information on an area desired by the user, only 
th local information inputted accompanied with the area code 
corresponding to the area previously inputted is decoded in the 
decoding part 114. 

Inputting an area is able by operating a control part 123 for 
example. The inputted area information is supplied to the decoding 
part 114 via an OR gate 122. 

Furthermore, the area input is also able using a global 
positioning system (GPS) system ("system" means that plural devic s 
are logically integrated and it is not referred to whether or not 
each of devices having individual configuration are contained in the 
same housing). That is, a radio wave from a GPS satellite is 
received by an antenna 118, and the received signal is supplied to a 
GPS receiving part 119. The GPS receiving part 119 calculates the 
position where the ISDB receiver of Fig. 11 has been installed from 
th received signal from the antenna 118 , and supplies thus obtain d 
position information (for example, its longitude and latitude or the 
like) to an area discriminating part 120. The area discriminating 
part 120 discriminates the area from the position information from 
th GPS rec iving part 119, and outputs a code assigned to that ar a 
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to a memory 121 for storing. Th code stored in the memory 121 is 
supplied to the decoding part 114 via the OR gate 122. In the 
decoding part 114, only the local information inputted accompanied 
with the area code agree with that code is decoded. 

The local information decoded in the decoding part 114 is 
supplied to a memory 115 and stored therein. Thus, in the memory 115, 
only the local information for the area where the user lives in, for 
x ample . 

When viewing the local information stored in the memory 115, 
th user operates so the control part 123. The local information is 
read out from the memory 115 corresponding to the operation of the 
control part 123 and supplied to a selector 116. In the selector 116, 
on or all of the local information from the memory 115 is selected 
and outputted via the selector 113. Thus, the local information is 
displayed on the display device 9 or printed out similarly to the 
above picture. 

Note that, even the local information of the same area, 
information necessary for the user and not so are mixed therein. In 
the selector 116, only the local information needed by the user is 
selected corresponding to the operation of the control part 123. 

In the selector 113, either the output of the decoding part 
112 or the output of the selector 116 is selected and outputted. 

Furthermore, in the ISDB receiver of Fig. 8, a time code is 
multiplexed as added information of video and audio accompanying with 
it, however, as the added information of video and audio accompanying 
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with that, for example, an identification cod to recognize a program 
composed of video and audio can be multiplexed other than that. In 
this case, a program corresponding to that identification code can be 
s lected by previously inputting the identification code of a desir d 
program. It enables, only when a desired program has transmitt d, 
the user to select and output the program. 

Note that, the details of the above ISDB transmitter and ISDB 
receiver have been disclosed in the patent application Nos. H7(1995)~ 
207158 and H7 (1995)-243453 previously filed in the present applicant, 
for example. 

Then, the adaptive decoding system 7, if the resolution of the 
decoded picture is lower than the resolution of the output device 
that will output the decoded picture among the display devices 9 and 
10, printer 11 and computer display 12, performs resolution creating 
processing to generate the decoded picture corresponding to the 
resolution of the output device by linearly coupling the decoded 
picture having low resolution to the prescribed coefficients. 

Note that, in the case where the number of pixels of the 
decoded picture is less than the number of pixels of the output 
device, for example, there is a method for matching the number of 
pix Is of the decoded picture with the number of pixels of the output 
d vice by performing an interpolation by means of an interpolation 
filter or the like. However, since such simple interpolation cannot 

e 

represent high frequency components not included in the original 
decod d pictur , it cannot improve the resolution. On the contrary, 
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in the resolution creating proc ssing, such high frequency components 
can be represented as described later. 

'Moreover, in the case where the number of pixels of the 
d coded picture is more than the number of pixels of the output 
d vice, the number of pixels of the decoded picture is reduced to 
coincide the number of pixels of the output device by thinning or 
r arranging the mean values of several pixels or the like as in the 
case of hierarchical coding* 

Fig. 12 shows a structural example of resolution creating 
circuit being a part of the adaptive decoding system 7 that performs 
th resolution creating processing. 

Also here, pictures having high resolution (of which the 
number of pixels is large) are defined as the lower hierarchy 
pictures and pictures having lower resolution (or which the number of 
pix Is is small) are defined as the upper hierarchy pictures 
similarly to the case of hierarchical coding. 

For example, in Fig. 13, if the parts shown by a dot are 
assumed as pixels forming the lower hierarchy picture (hereinafter, 
it is preferably referred to as lower pixel) and the parts shown by a 
circle are assumed as pixels forming the upper hierarchy picture 

X 

(hereinafter, it is preferably referred to as upper pixel), the 
resolution creating circuit converts the upper hierarchy picture 
formed of the pixels shown by the circles into the lower hierarchy 
picture formed of the pixels shown by dots. 

That is, the upper hi rarchy picture is supplied to a 
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classif icatory blocking circuit 131 and a blocking circuit for 
calculation of predictive value 133. 

The classificatory blocking circuit 131 forms classif icatory 
blocks that include the prescribed marked pixel from the upper 
hierarchy picture supplied thereto. More specifically, the 
classificatory blocking circuit 131 forms classificatory blocks each 

which is formed of 5X5 (horizontal by longitudinal) upper pixels and 
has the marked pixel in its center as shown by enclosing with a solid 
line in Fig. 13, for example. 

Hereinafter, the 5X5-upper pixels which form the 
classificatory block (the part shown by circles in Fig. 13) is 
preferably denoted as follows: the upper pixel locating at the i-th 
from the left and the j-th from the top in the classificatory block 
is denoted by B^. Therefore, in the embodiment of Fig. 13, the 
classificatory block which has an upper pixel B 33 as the marked pixel 
is formed. The lower pixel generated (predicted) from the upper 

r 

pix Is forming the classificatory block (the part shown by a dot in 
Fig. 13) is preferably denoted by A Aj similarly to the upper pixel, 
hereinafter. 

If forming the classificatory block, the classificatory 
blocking circuit 131 outputs it to a classifying circuit 132. The 
classifying circuit 132 classifies the classificatory block into 
prescribed classes according to its property, and supplies thus 
obtained class information to a forecasting circuit 134. 

To the forecasting circuit 134, a predictive value calculating 



41 



block is further supplied from the blocking circuit for calculation 
of predictive value 133 . In the blocking circuit for calculation of 

predictive value 133 , for example, 3X3 pixels of predictive value 
calculating block centering the marked pixel B 33 , as shown by 
enclosing with a dotted box, is formed and supplied to the 
forecasting circuit 134. 

Here, the method for forming the predictive value calculating 
block and classif icatory block is not only limited to the above. 
Note that, the form of the predictive value calculating block is 
basically free, however, the classif icatory block is preferable to 
include the characteristics of the predictive value calculating block. 

If receiving the predictive value calculating block and the 
class information of the marked pixel, the forecasting circuit 134 
p rforms adaptive processing to obtain the predicted value of the 
pixel value of the lower pixels by linearly coupling predictive 
co fficients described later corresponding to the received class 
information to the pixel values of the upper pixels forming the 
predictive value calculating block. More specifically, for example, 
th forecasting circuit 134 obtains the predicted values of lower 

pixels A 43 , A^, A 45 , As3, As 4 , A^, A^, A^ and in the area of 3X3 
pixels centering the marked pixel B 33 from predictive coefficients 
corresponding to the class and upper pixels forming the predictive 
value calculating block B 22 , B 23 , B 24 , B 32 , B 33 , B 34 , B 42 , B 43 and B 44 . 

In the forecasting circuit 134, hereinafter, the similar 
processing is sequentially perform d with setting all the upper 
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pixels exc pt for the pixel B 33 as the marked pix Is. As a result, 
th predicted values of all the lower pixels forming the lower 
hierarchy picture are obtained . 

Here, to the forecasting circuit 134, a resolution signal is 
supplied from a resolution setting circuit 135. The resolution 
setting circuit 135, for example, communicates with the output d vie 
to which the picture is outputted from the adaptive decoding system 7 
(in the embodiment of Fig. 1, one of the display devices 9 and 10, 
printer 11 and computer display 12) and recognizing its resolution, 
and supplies the resolution signal representing the recognized 
resolution to the forecasting circuit 134. 

In the forecasting circuit 134, the predictive coefficients of 
pictures having various resolutions have stored, so that a lower 
hierarchy picture can be obtained using the predictive coefficients 
of a resolution corresponding to the resolution signal from the 
resolution setting circuit 135. 

In addition, it is able to input a resolution to the 
resolution setting circuit 135 by operating an operating part (not 
shown) for example, other than communicating with the output devic . 

Hereinafter, classifying processing in the classifying circuit 
132 and adaptive processing in the forecasting circuit 134 will b 
described. 

First, it will be described about the classifying processing. 
As shown in Fig. 14A, now it is assumed that a block composed 
of 2X2 pixels (classif icatory block) is formed of a certain mark d 
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pix 1 and thr e adjac nt pixels, and each pix 1 is repr sent d by one 
bit (which has either level 0 or level 1). In this case, as shown in 

Fig, 14B, the block of 2X2 pixels, four pixels, can be classifi d 
into 16 (= (2 1 ) 4 ) patterns based on a level distribution of each 
pixel* Such classification by patterns is the classifying processing. 

Note that, the classifying processing can be performed also 
considering the activities (complexity of picture and sudden chang ) 
or the like, of the picture (picture in the block). 

Here, e.g., eight bits degree are normally assigned to each 
pixel. Moreover, in this embodiment, the classificatory block is 

composed of 5X5 pixels, 25 pixels as described above. Therefore, if 
the classifying processing is executed on such classificatory blocks, 
th y will be classified into an enormous number of classes that is 

To cope with that, the classifying circuit 132 is able to 
x cute the ADRC processing on the classificatory blocks previous to 
th classifying processing. By executing the ADRC processing, the 
number of bits of the pixels forming a classificatory block can be 
r duced, and the number of classes can be reduced. 

Next, the adaptive processing will be described. 

For example, it is now considered that the predictive value 
E[y] of the pixel value (y) of a lower pixel is obtained based on a 

"i * 

lin ar primary combined model provided by linearly coupling the pixel 
values of several upper pixels (hereinafter, they are preferably 
ref rr d to as learning data) x lf x 2 , ... to the prescribed 
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predictive coeffici nts w lf w 2 , .... In this case, the predictiv 
value E[y] can be represented by the following equation: 



E[y] = w A x x + w 2 x 2 + 



...(1) 



To generalize that, a matrix W being a set of the predictive 
coefficients (w) , a matrix X being a set of the learning data and a 
matrix Y 1 being a set of the predicted values E[y] are defined by the 
following equations: 



r 



X = 



X I1 X 12 



v. 



•In 



X 2 x X 22 ••• X 2n 









r ~\ 


w = 

■ 


w 2 

... 


, Y' = 


Etyj 

... 



As a result, the following observation equation is obtain d 



XW « Y f 



...(2) 



And it is consid red to obtain pr diet d values E[y] 
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approximate to th pixel values (y) of th lower pix Is applying a 
1 ast square to this observation equation. In this case, a matrix Y 
being a set of the pixel values of the lower pixels (hereinafter, 
th y are preferably referred to as teaching data) (y) and a matrix E 
b ing a set of the residuals (e) of the predicted values E[y] to th 
pix 1 values (y) of the lower pixels are defined by the following 
equations : 



E = 



r ^ 

Yi 



J 



* • 



Ym 



As a result, the following residual equation comes from the 



quation (2). 



XW = Y + E • • • (3) 

In this case, predictive coefficients w ± to be used to obtain 
the predictive values E[y] approximate to the pixel values (y) of the 
lower pixels can be obtained by minimizing the square error such as 
th following: 
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Thus, it can be said that when the solution by that the above 
square error has differentiated by the predictive coefficient w ± 
becomes 0, that is, the optimum value to obtain the predictive values 
E [y] approximate to the pixels (y) of the lower pixels is the 
predictive coefficients w A satisfying the following equation: 



3ei 3e* de« 

+ e* + • . . + e« = 0(x - 1,2, . . . , n) 



i 

dWi dWi dWi 



...(4) 



First, the equation (3) is differentiated by the predictive 
coefficient w A , and obtaining the following equations: 



de± det dei 

— Xil, — Xl 2, • • . * = XlnlX = X, Af • • • m III) 

Urn- *V_ _ * »V_ _ ^ 

OWi OWa OWn 



...(5) 



The equations (4) and (5) give the following equation: 



m m 



V eiXii « 0, V eiXi 2 = 0, . . . , ^ eiXln = ® 



i-1 i-1 i-1 

...(6) 



Considering the relationship among th learning data (x), 
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predict iv co f f icient (w) , teaching data (y) and residual ( ) in the 
r siduals equation of the equation (3), the following normal 
quations can be obtained from the equation (6). 



XiaXnjwx + XmXiajwi + . . . + XinXm^Wn « Xi B yij 



V 



...(7) 



The normal equations of the equation (7) can be formed by the 
same number as the number of the predictive coefficients (w) to be 
obtained. Therefore , the optimum predictive coefficient (w) can be 
obtained by solving the equation (7): however, to solve the equation 
(7), it is needed that the matrix formed of coefficients concerning 
with the predictive coefficient (w) is regular). Note that, to solve 
the equation (7), e.g., a sweeping method (elimination of Gauss- 
Jordan) or the like can be applied. 

As the above, in the adaptive processing, the optimum 
predictive coefficient (w) is previously obtained for each class, and 
the predictive values E[y] approximate to the pixel values (y) of the 
lower pixels are obtained using that predictive coefficient (w) by 
the equation (1). This adaptive processing will be performed in the 
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forecasting circuit 134. 

For example, it is assumed that the above learning is now 
performed on the classes obtained by the classification of the 

classif icatory blocks formed of 5X5 pixels enclosed with a solid 
line in Fig. 13, and predictive coefficients w 1 (A 43 ) to w 9 (A« 3 ), w^A^) 
to 14 (Am), W^A^) to w 9 (A 45 ), w 1 (A S3 ) to w 9 (As 3 ) , w 1 (A S4 ) to w 9 (As 4 ), w^A^) 
to w 9 (As 5 ), w 1 (A 63 ) to w^A^), w 1 (A 64 ) to w 9 (A 64 ) and w 1 (A 65 ) to w 9 (Ae 5 ) are 
r spectively obtained, to respectively obtain the predictive values 
E[A 43 ], E[A^], E[A„], E[A5 3 ] , E[A S4 ], E [A^] , E[A, 3 ] , EfA^] and E^) of 
lower pixels in a predictive value calculating block, A^, A^, A^, Ag 3 , 
Ag 4 , As 5/ ^63/ and Aes* In this case, in the forecasting circuit 134, 
the predictive values E[A 43 ], E [A 44 ] , E[A 45 ] , E[As 3 ] , E[As 4 ], E[As 5 ] , 
E[A6 3 ] , E[A 64 ] and E[A 65 ] of the HD pixels in the predictive value 
calculating block, A^, A 44 , A 45 , As 3 , Ag 4 , A55, ^3, A^ and A 65 are 
respectively obtained according to the following equations 
corresponding to the equation (1). 

ECA^] = w 1 (A 43 )B 22 + w 2 (A 43 )B 23 + w 3 (A 43 )B 24 

+ w 4 (A 43 )B 32 + w 5 (A 43 )B 33 + w 6 (A 43 )B 34 

+ w 7 (A 43 )B 42 + w 8 (A 43 )B 43 + w 9 (A 43 )B 44 
E[A 44 ] = W 1 (A 44 )B 22 + w 2 (A 44 )B 23 + w 3 (A, 4 )B 24 

+ w 4 (A 44 )B 32 + w 5 (A 44 )B 33 + w 6 (A 44 )B 34 

+ w 7 (A 44 )B 42 + w 8 (A 44 )B 43 + w 9 (A 44 )B 44 
EIA^] = W 1 (A 45 )B 22 + w 2 (A 45 )B 23 + w 3 (A 45 )B 24 

+ w 4 (A 45 )B 32 + w 5 (A 45 )B 33 + w 6 (A 45 )B 34 
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+ W I (A 4S )B M + 1*8(^5)843 + W 9 (A 45 )B 44 
E[As 3 ] - w 1 (A S3 )B J2 + w a (As3)B 23 + w 3 (A S3 )B :i4 

+ W 4 (A S3 )B 3:I + W s (As 3 )B 33 + w 6 (As 3 )B 34 

+ W 7 (As 3 )B 4a + W 8 (As 3 )B 43 + W 9 (As 3 )B 44 
E[As 4 ] = W 1 (A S4 )B 22 + W 2 (As 4 )B 23 + W 3 (As 4 )B 24 

+ w 4 (As 4 )B 32 + w 5 (As 4 )B 33 + w 6 (As 4 )B 34 

+ W 7 (As 4 )B 42 + W 9 (As 4 )B 43 + W 9 (A5 4 )B 44 
EJAsj] = w 1 (A S5 )B 22 + W 2 (A„)B a , + w 3 (As S )B 24 

+ w 4(Ass)B 32 + W 5 (As S )B 33 + w 6 (As S )B 34 

+ W 7 (A S5 )B 42 + W 8 (A SS )B 43 + w 9 (As 5 )B 44 
E[A,i 3 ] = W 1 (A 63 )B 22 + W 2 (A 63 )B 23 + W 3 (A 63 )B 24 

+ W4(A 63 )B 32 + WgCA^B,, + W 6 (I^ 3 )B 34 , 

+ W 7 (A 63 )B 42 + W 8 (A 63 )B 43 + W 9 (A* 3 )B tt 
EtA^] - W 1 (A 64 )B 22 + W 2 (A 64 )B 23 + W 3 (A 64 )B 24 

+ W4(A,i 4 )B 32 + W 5 (A 64 )B 33 + W 6 (A 64 )B 34 

+ w 7 (A 64 )B 42 + w 8 (A 64 )B 43 + w,(A„)B M 
EtAs! = w 1 (A 65 )B 22 + W^A^B,, + w^s^ 

+ w 4 (^5)8,3 + w 5 (^5)633 + w^A^Ba, 

+ *M ^5)643 + W„ (^5)643 + W 9 (^5)644 

...(8) 

Note that, the adaptive processing differs from interpolation 
processing from the viewpoint of that components included in only the 
lower hierarchy picture are represented. That is, it can be said 
'that th adaptive processing is the same as the interpolation 



proc ssing using so-call d interpolation filter, so far as only the 

quations (1) and (8) are viewed. However , since the predictive 
coefficients (w) corresponding to the tap coefficients of the 
interpolation filter are obtained by using the teaching data (y) 
quasi learning, also the components included in the lower hierarchy 
picture can be represented. From that reason, the adaptive 
processing can be said that quasi a processing having a resolution 
creating function. 

The detail of the adaptive processing has been disclosed in 
the publication of the patent laid-open No. H5 (1993) -328185 
pr viously filed by the present applicant, for example. 

Now, Fig. 15 shows a structural example of the forecasting 
circuit 134 of Fig. 12. 

The class information from the classifying circuit 132 are 
supplied to a switch 141. The switch 141 selects one of terminals al 
to a4 corresponding to the resolution signal from the resolution 
setting circuit 135. The terminals al to a4 of the switch 141 ar 
connected to the address terminals (AD) of read only memories (ROM) 
143 to 146 respectively. Thereby, the class information from th 
classifying circuit 132 is supplied to one of the ROMs 143 to 146 via 
th switch 141 as an address. 

A switch 142 selects one of terminals bl to b4 corresponding 
to the resolution signal from the resolution setting circuit 135 
similarly to the switch 141. The terminals bl to b4 are connect d to 
the data terminals D of the ROMs 143 to 146 respectively. Note that, 
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the switch 142 int rlocks the switch 141, and for example, if th 
switch 141 selects one of the terminals al to a4, the switch 142 
sel cts corresponding one of terminals bl to b4. 

In the ROMs 143 to 146, the predictive coefficients for ach 
class obtained by the learning described above have been stored in an 
address corresponding to the class. Specifically, in the ROMs 143 to 
146, the predictive coefficients to be used to convert, for example, 
if the aforementioned hierarchical coding is performed , the picture 
of the most upper hierarchy into a lower hierarchy picture 
corresponding to the resolution of the display device 9 or 10, 
printer 11 or computer display 12 have been stored respectively. 

To an arithmetic circuit 147, the predictive value calculating 
block is supplied from the blocking circuit for calculation of 
predictive value 133, and the predictive coefficient is supplied from 
one of the ROMs 143 to 146 is supplied via the switch 142, 
respectively. The arithmetic circuit 147 performs a sum of products 
operation corresponding to the equation (1) or (8) using the 
predictive value calculating block and the predictive coefficient, 
and obtaining the lower hierarchy picture corresponding to the 
r solution of the output device. 

In the forecasting circuit 134 constructed as the above, one 
of the terminals al to a4 is selected by the switch 141 corresponding 
to the resolution signal from the resolution setting circuit 135, and 
also a corresponding one of the terminals bl to b4 is selected by the 
switch 142 with interlocking it. 



52 



* 



Th n, th class information from the classifying circuit 132 
is supplied to the address terminal of one of the ROMs 143 to 146 
that has been connected to the terminal selected by the switch 141 
(on of the terminals al to a4) (hereinafter, it is preferably 
ref rred to as selected ROM) . In the selected ROM, the predictiv 
coefficient stored in the address corresponding to the class suppli d 
to its address terminal is read out and outputted from its data 
t rminal. 

Since the switch 142 interlocks the switch 141 as described 
above, the predictive coefficient read out from the selected ROM is 
supplied to the arithmetic circuit 147 via the switch 142. 

To the arithmetic circuit 147 , the predictive value 
calculating block is supplied from the blocking circuit for 
calculation of predictive value 133 in addition to the predictive 
coefficient, as described above. In the arithmetic circuit 147, the 
sum of products operation corresponding to the equation (1) or (8) is 
p rformed using the predictive value calculating block and the 
pr dictive coefficient supplied thereto, so that the lower hierarchy 
picture corresponding to the resolution of the output device is 
generated and outputted. 

Thus , the user can view the picture corresponding to the 
output device. 

Note that, in the above embodiment, the predictive 
co f f icients have stored in the adaptive decoding system 7 . However, 
the predictiv coefficients may be transmitted from the transmitting 
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sid as th information nec ssary for decoding. Also methods for 
forming the classif icatory block or the predictive value calculating 
block can be instructed from the transmitting side. 

According to the video encoder according to Claim 1 and the 
method for encoding a picture according to Claim 5, the one or more 
processing among the plural processing necessary to encode the 
picture considering the other processing. Thereby, efficient 
processing can be performed. 

According to the video decoder according to Claim 6 and the 
method for decoding a picture according to Claim 7 , a decoded picture 
corresponding to the resolution of an output device to which the 
picture is outputted is generated by linearly coupling transmit data 
to prescribed coefficients. Thereby, it can cope with output devices 
having various resolutions. 

According to the video processor according to Claim 8 and the 
method for processing a picture according to Claim 9, one or mor 
processing among the plural processing necessary to encode the 
picture considering the other processing, and on the other hand, a 
d coded picture corresponding to the resolution of an output device 
to which the picture is outputted is generated by linearly coupling 
thus obtained data to prescribed coefficients. Thereby, efficient 
processing can be performed and also it can cope with output devic s 
having various resolutions. 

Industrial Capability 
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The present invention is applicable to a video processing 
system including a video camera, ATV, video editor, etc., which deals 
with video data having a standard resolution, high resolution and low 
r solution. 
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